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ABSTRACT 

This  study  evaluated  telemedicine  use  in  remote  military  treatment  facilities  (MTFs)  ashore 
over  a  4-month  period  to  help  guide  telemedicine  applications  for  shipboard  medical  de¬ 
partments.  A  prospective  study  design  was  used  to  evaluate  specialty  care  provided  by  an 
ear/nose/throat  (ENT)  physician  via  videoconferencing  (VC)  for  patients  at  remote  MTFs  in 
TRICARE  Region  9.  The  study  provided  a  complete  and  continuous  sample  of  ENT  consul¬ 
tations  during  a  planned  4-month  period.  Data  sources  included  a  telemedicine  database  and 
telephone  interviews  to  assess  attitudes  of  physician  and  nonphysician  medical  personnel.  A 
total  of  193  VC  consultations  (hereafter  referred  to  as  teleconsultations)  were  conducted  fol¬ 
lowing  referrals  from  primary  providers.  Patients  were  mostly  young,  male,  active-duty  per¬ 
sonnel.  Forty-five  percent  of  the  193  teleconsultations  resulted  in  changed  diagnosis  by  the 
ENT  specialist  relative  to  initial  diagnosis  by  the  referring  provider.  This  rate  of  clinical  im¬ 
pact  was  substantial,  and  it  generalized  across  various  ENT  conditions,  demographics,  and 
MTFs.  Medical  personnel  reported  generally  positive  attitudes  about  telemedicine  technolo¬ 
gies  and  the  telemedicine  process  in  TRICARE  Region  9.  Nonphysician  providers  reported 
slightly  more  favorable  attitudes  compared  to  physicians.  These  results  suggest  that  ENT 
telemedicine  has  substantial  clinical  impact  in  the  military  populations  treated  at  MTFs.  A 
high  rate  of  changed  diagnoses  (45%)  was  observed  across  age,  gender,  military  status,  ENT 
conditions,  and  treatment  facilities.  Medical  personnel  reported  positive  attitudes  about  us¬ 
ing  the  telemedicine  system.  These  results  support  the  use  of  telemedicine  in  shipboard  med¬ 
ical  departments. 


INTRODUCTION 

IT  HAS  BECOME  INCREASINGLY  CLEAR  that  effec¬ 
tive  application  of  telemedicine  technologies 
such  as  email,  the  Internet,  and  live  videocon¬ 
ferencing  (VC)  is  essential  for  the  U.S.  armed 
forces  to  optimize  their  health  care  mission.1-4 
This  is  especially  true  for  personnel  deployed 
or  stationed  in  remote  environments.5'6  Tele¬ 


medicine  can  provide  timely  access  to  special¬ 
ized  care  for  patients  and  medical  personnel  on 
ships  at  sea  with  limited  medical  resources.6 
Telemedicine  provides  many  benefits,7-9  which 
extend  to  personnel  and  their  dependents 
treated  at  Military  Treatment  Facilities  (MTFs) 
in  medically  underserved  regions  ashore.10'11 

This  study  evolved  from  a  naval  research  di¬ 
rective  to  determine  how  telemedicine  could 


'Naval  Health  Research  Center,  Field  Medical  Technologies,  San  Diego,  California. 
2Naval  Medical  Center,  San  Diego,  California. 

3MTS  Technologies,  Inc.,  Arlington,  Virginia. 
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support  shipboard  medical  departments.1 
Larger  ships,  such  as  carriers  and  amphibious 
ships,  have  extensive  medical  departments,  in¬ 
cluding  physician  and  nonphysician  personnel, 
to  perform  the  functions  of  a  small  hospital 
(e.g.,  surgery).  Some  ships  have  telemedicine 
technologies  such  as  the  Internet,  VC,  and 
computed  radiology  installed.12  Unfortunately, 
the  few  evaluation  studies  of  shipboard  tele¬ 
medicine  processes  that  exist  are  limited  to 
these  large  ships.13  No  organized  telemedicine 
capabilities  currently  exist  on  small  ships,  such 
as  cruisers  and  destroyers,  to  assist  medical  de¬ 
partments  headed  by  nonphysicians,  such  as 
independent  duty  corpsmen  (IDCs).6  Evaluat¬ 
ing  telemedicine  networks  currently  used  at 
MTFs  ashore  provides  an  opportunity  to  study 
telemedicine  use  by  military  medical  personnel 
and  patient  populations;  results  may  general¬ 
ize  to  help  guide  applications  for  shipboard 
medical  departments.  Some  have  pointed  out 
that  the  field  of  telemedicine  in  general  needs 
more  systematic  evaluation  studies.14'15 

This  study  followed  up  on  a  retrospective 
evaluation  of  the  first  2.5  years  of  telemedicine 
network  operation  in  TRICARE  Region  9.11  In 
Region  9,  remote  MTFs,  such  as  Fort  Irwin  and 
Port  Hueneme,  access  specialty  care  by  physi¬ 
cians  at  Naval  Medical  Center,  San  Diego  via 
live  VC.  The  retrospective  study  showed  that 
telemedicine  use  increased  with  time  (since  op¬ 
erational)  at  various  MTFs  and  the  duration  of 
operation  of  individual  telemedicine  special¬ 
ties,  e.g.,  ear/nose/throat  (ENT).  In  addition, 
the  retrospective  data  suggested  that  approxi¬ 
mately  half  of  ENT  consultations  had  a  clinical 
impact,  measured  by  change  in  diagnosis. 

The  present  study  extended  the  retrospective 
study  of  Region  9  telemedicine  in  two  ways:  (1) 
Analysis  focused  on  determining  a  reliable  rate 
of  clinical  impact,  namely  whether  diagnosis 
changed,  for  ENT  telemedicine  over  a  planned 
4-month  study  period.  ENT  conditions  can 
pose  a  concern  for  shipboard  medical  depart¬ 
ments,16-18  and  little  evaluation  exists  of  ENT 
teleconsultations  in  MTFs.19'20  The  generality 
of  these  procedures  for  different  ENT  condi¬ 
tions  and  patient  populations  is  unclear.  (2)  In¬ 
terviews  with  military  medical  personnel,  e.g., 
general  medical  officers  (GMOs),  IDCs,  and 


medical  technicians,  were  conducted  to  assess 
their  attitudes  and  experiences  with  Region  9 
telemedicine.  Some  evidence  indicates  that 
telemedicine  attitudes  (e.g.,  satisfaction)  and 
experience  are  positively  correlated  with 
telemedicine  use.21  Thus,  satisfaction  with 
telemedicine  and  experiences  with  telemedi¬ 
cine  and  related  technologies  were  measured 
among  physician  and  nonphysician  medical 
personnel. 

METHODS 

This  was  a  prospective  evaluation  of 
telemedicine  teleconsultations  that  were  con¬ 
ducted  by  an  ENT  specialty  physician.  The  data 
were  extracted  from  a  telemedicine  database 
called  the  referral  management  system  (RMS) 
and  from  structured  interviews  with  physician 
and  nonphysician  medical  personnel.  The 
study  was  conducted  over  a  planned  4-month 
period  and  included  patient  consultations  and 
medical  personnel  at  remote  MTFs  in  TRICARE 
Region  9  and  Lemoore  Naval  Air  Station. 

Data  sources  and  subjects 

Observations  came  from  two  sources:  (1)  A 
database  of  telemedicine  consultations  and  (2) 
structured  interviews  with  medical  personnel. 

Referral  management  system  database 

The  telemedicine  consultations  (n  =  193)  oc¬ 
curred  between  November  1,  2000,  and  March 
1,  2001,  and  were  extracted  from  the  database. 
These  were  all  sessions  in  which  primary 
providers  referred  patients  for  consultations 
with  an  ENT  specialist  for  initial  evaluation. 
These  consultations  were  provided  for  patients 
treated  at  treatment  facilities  in  TRICARE  Re¬ 
gion  9  and  at  Lemoore  Naval  Air  Station.  These 
facilities  comprise  the  functional  telemedicine 
network  centered  at  NMCSD.11  Individual  pa¬ 
tients  contributed  one  or  more  observations  for 
one  or  more  medical  conditions. 

Medical  personnel 

Medical  personnel  from  the  Region  9 
telemedicine  network  completed  telephone  in- 


TELEMEDICINE  AND  MILITARY  SPECIALTY  CARE 


303 


terviews.  These  were  physicians  ( n  =  8)  and 
nonphysicians  (n  =  7),  11  males  and  4  females. 
The  physicians  consisted  of  2  GMOs,  3  primary 
care  physicians,  and  3  specialty  physicians 
(child  psychiatry,  ENT  and  neurology).  The 
nonphysicians  consisted  of  1  IDC,  4  medical 
technicians,  1  nurse,  and  1  physician's  assis¬ 
tant.  Of  the  15,  12  were  on  active  duty  and  3 
were  civilians.  This  was  a  targeted  sample 
based  on  advice  of  the  telemedicine  coordina¬ 
tor  in  TRICARE  Region  9,  who  identified  those 
who  worked  on  these  telemedicine  consulta¬ 
tions.  Over  90%  of  those  targeted  were  con¬ 
tacted  and  completed  the  interview.  A  few  mil¬ 
itary  personnel  had  transferred  to  another 
assignment  and  were  unavailable.  There  were 
no  more  than  a  few  telemedicine  personnel  at 
each  clinic  and  at  least  1  and  as  many  as  4  med¬ 
ical  personnel  surveyed  worked  in  each  of  8 
MTFs.  There  were  approximately  25  telemedi¬ 
cine  personnel  working  in  Region  9  telemedi¬ 
cine  during  the  study  period.  An  exact  number 
is  difficult  to  determine  due  to  turnover  and 
part  time  assignments. 

TRICARE  region  9  telemedicine  network 

TRICARE  provides  healthcare  for  active-duty 
personnel,  their  qualified  family  members. 
Civilian  Health  and  Medical  Program  of  the 
Uniformed  Services-eligible  retirees,  and  the 
families  and  survivors  of  all  uniformed  services. 

The  telemedicine  network  in  southern  Cali¬ 
fornia  includes  8  MTFs  [Naval  Medical  Center, 
San  Diego,  Port  Hueneme,  Fort  Irwin,  Edwards 
Air  Force  Base  (AFB),  Vandenburg  AFB,  Twen- 
tynine  Palms,  Los  Angeles  AFB,  Lemoore 
NAS],  most  with  limitations  on  medical  spe¬ 
cialization.  NMCSD  functions  as  the  "hub"  of 
the  telemedicine  network.  The  remote  MTFs 
are  the  "spokes"  at  various  distances  from 
NMCSD  (121-320  miles).  Although  the 
Lemoore  facility  is  not  part  of  Region  9,  it  was 
included  because  it  is  part  of  the  functional 
telemedicine  network  centered  at  NMCSD.  The 
spoke  facilities  received  telemedicine  consulta¬ 
tions  from  the  ENT  specialist  at  the  hub  in  San 
Diego.  Such  a  "hub-and-spoke"  telemedicine 
network  model  has  been  applied  frequently  in 
recent  years  in  many  settings.8'22 


Referral  management  system 

Telemedicine  consultations  were  managed 
and  recorded  via  a  secure  Web-based  applica¬ 
tion  installed  for  Region  9.  When  a  primary 
provider  at  a  remote  facility  determined  that  a 
patient  required  specialty  consultation,  the  pa¬ 
tient  was  referred  to  a  telemedicine  coordina¬ 
tor.  If  the  specialty  care  was  not  available  at  the 
patient's  site,  the  telemedicine  coordinator 
(e.g.,  medical  technician)  scheduled  a  telecon¬ 
sultation  with  the  ENT  physician.  The  patient 
at  the  remote  site  then  was  registered  in  the  sys¬ 
tem  along  with  information  about  background, 
symptoms,  and  primary  provider  diagnosis. 
The  ENT  specialist  at  San  Diego  then  accessed 
the  referral  management  system  for  scheduled 
consultations  and  patient  case  information. 

During  the  consultation  itself,  the  telemedi¬ 
cine  coordinator  presented  the  patient  to  the 
consultant  via  VC  and  executed  relevant 
ENT  examination  procedures.  Following  the 
telemedicine  appointment,  the  consultant  en¬ 
tered  comments  on  diagnosis  and  treatment  in 
the  patient's  record. 

The  choice  of  ENT  specialty  for  this  study 
was  based  on  the  fact  that  such  consultations 
were  the  most  common  in  TRICARE  Region 
9.11  Initial  evaluations  were  chosen  because 
these  are  the  sessions  where  the  primary 
provider  and  specialist  determine  diagnosis.  In 
this  study,  store-and-forward,  preoperative  or 
postoperative,  or  follow-up  sessions  were  ex¬ 
cluded  because  these  sessions  usually  did  not 
function  to  determine  diagnoses.  The  ENT 
telemedicine  examinations  depended  on  real¬ 
time  videoconferencing,  unlike  other  special¬ 
ties.  For  instance,  dermatology  makes  exten¬ 
sive  use  of  the  store-and-forward  process  such 
as  emailing  of  digital  pictures  to  determine  di¬ 
agnosis.3 

The  following  variables  were  extracted  from 
the  database: 

•  Patient  demographics  (age,  gender,  and  mil¬ 
itary  status) 

•  Reason  for  consultation  (initial,  preopera¬ 
tive,  postoperative  follow-up,  other) 

•  Patient  and  primary  provider  site 

•  Consultant  medical  specialization  (ENT, 

neurology,  psychiatry,  rheumatology) 


304 


MELCER  ET  AL. 


•  Primary  provider  and  consultant  diagnosis 
(provider  and  consultant  comments  were 
coded  into  ICD-9  categories) 

•  Telemedicine  modality  (VC  or  store-and-for- 
ward) 

The  ENT  consultations  included  a  diversity 
of  problems.  All  cases  were  assigned  to  one  of 
the  following  five  categories:  hearing  (e.g.,  tin¬ 
nitus),,  sleep-related  (e.gv  sleep  apnea),  tonsil- 
related,  upper  respiratory  (e.g.,  sinusitis),  and 
integumentary  (e.g.,  skin  cyst).  These  cate¬ 
gories  were  chosen  in  consultation  with  the 
lead  ENT  specialist.  Thus,  it  was  possible  to  see 
the  relative  frequency  of  major  ENT  conditions 
in  this  population,  and  to  determine  variation 
in  clinical  impact  of  teleconsultations  among 
these  types  of  cases. 

Interviews  with  medical  personnel 

The  medical  personnel  completed  brief  tele¬ 
phone  interviews,  approximately  10  minutes 
in  length,  to  determine  their  medical  back¬ 
grounds  and  experience  with  technology,  and 
to  assess  their  attitudes  toward  telemedicine. 
These  interviews  were  conducted  at  the  end 
of  the  study  period  according  to  a  structured 
questionnaire  format.  The  questions  and  inter¬ 
view  forms  were  based  on  those  used  in  pre¬ 
vious  work  at  Naval  Health  Research  Center.13 
The  format  was  influenced  by  Dillman's23 
methodology  for  mail  questionnaires. 

The  following  variables  were  recorded  from 
the  interview  forms: 

•  Medical  positions  (e.g.,  IDC,  nurse,  physi¬ 
cian)  and  specialization  (e.g.,  dermatology, 
psychiatry) 

•  Years  experience  with  telecommunications 
technologies  (e.g.,  VC,  Internet,  e-mail) 

•  Medical  purpose  for  telemedicine  use  (e.g., 
confirm  diagnosis,  patient  education) 

•  Attitudes  were  assessed  using  a  5-point  Lik¬ 
ert  rating  scale  (e.g.,  1  =  very  satisfied,  2  = 
somewhat  satisfied,  3  =  undecided,  4  = 
somewhat  dissatisfied,  5  =  very  dissatis¬ 
fied).  Five  to  7  questions  were  used  to  assess 
each  of  four  different  attitudinal  variables: 

•  Satisfaction  with  technologies 

•  Satisfaction  with  modalities  (e.g.,  sound,  im¬ 
ages) 


•  Usefulness  of  technologies 

•  Perception  of  telemedicine  (e.g.,  "Telemedi¬ 
cine  allows  me  to  give  better  health  care  to 
my  patients"). 

Data  analysis 

The  data  analysis  addressed  the  following 

questions: 

•  What  percentage  of  teleconsultations  had 
clinical  significance,  namely  a  difference  in 
diagnosis  between  the  primary  provider  and 
the  ENT  specialist? 

•  Which,  if  any,  demographic  or  patient  con¬ 
ditions  were  associated  with  changed  diag¬ 
noses? 

•  What  were  attitudes  and  experiences  of 
medical  personnel  regarding  telemedicine? 
Were  these  attitudes  and  experiences  related 
to  telemedicine  use?  Did  attitudes  vary  by 
level  of  medical  training  (i.e.,  physician  or 
nonphysician)? 

Chi-square  tests  (p  <  0.05)  were  conducted 
to  test  for  the  significance  of  associations  be¬ 
tween  change  in  diagnosis  and  ENT  condition, 
gender,  and  military  status.  The  sample  sizes 
for  other  variables  were  too  small  to  permit  this 
test. 

Descriptive  statistics  are  presented  for  the  in¬ 
terview  data.  Preliminary  trends  are  described 
for  the  interview  data  because  of  the  small  sam¬ 
ples  of  8  physicians  and  7  nonphysician  med¬ 
ical  personnel.  Thus,  the  comparisons  between 
physicians  and  nonphysicians  should  be  inter¬ 
preted  with  caution. 


RESULTS 

A  total  of  193  ENT  teleconsultations  were 
conducted  following  referrals  from  primary 
providers.  Forty-five  percent  of  these  consulta¬ 
tions  led  to  a  change  in  diagnosis  by  an  ENT 
specialty  physician  relative  to  the  initial  diag¬ 
nosis  by  the  referring  provider.  This  rate  of 
change  generalized  across  different  ENT  prob¬ 
lems,  age,  gender,  military  status,  and  facility 
site. 

Medical  personnel  reported  generally  posi¬ 
tive  attitudes  toward  telemedicine  technologies 
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and  the  telemedicine  process  in  TRICARE  Re¬ 
gion  9.  Compared  with  physicians,  nonphysi¬ 
cians  reported  slightly  more  favorable  attitudes 
and  used  telemedicine  more  often  and  in  more 
ways. 

Demographics 

The  sample  consisted  of  approximately  70% 
active-duty  adult  males  between  18  and  44 
years  old  (Table  1)  and  30%  females  and  mili¬ 
tary  dependents.  Over  75%  of  the  consultations 
came  from  the  Lemoore  or  Port  Hueneme  fa¬ 
cilities.  An  interruption  in  certification  of  the 
ENT  specialist  by  Fort  Irwin  and  Edwards  AFB 
limited  ENT  telemedicine  at  these  sites.  The  de¬ 
mographic  variables  will  be  considered  for  pos¬ 
sible  associations  with  rate  of  diagnosis  change. 

Clinical  impact 

Almost  half  (45%)  of  the  consultations  pro¬ 
duced  a  change  in  patient  diagnosis  by  the  ENT 
specialist  relative  to  the  initial  diagnosis  of  the 
primary  provider  (Fig.  1). 

Because  the  ENT  cases  included  diverse 
medical  problems  (e.g.,  hearing-  and  sleep-re- 


Table  1.  Demographics  for  Patients  Referred  for  ENT 
Consultations  via  VC 


Total  consults 

n  =  193 

%  =  100 

Gender 

Male 

133 

68.9 

Female 

58 

30.1 

Unknown 

2 

1.0 

Age,  years 

1-10 

19 

9.8 

11-17 

5 

2.6 

18-24 

50 

25.9 

25-44 

92 

47.7 

45-65 

19 

9.8 

65+ 

7 

3.6 

Unknown 

1 

.5 

Military  status 

Active  duty 

132 

68.4 

Dependent 

59 

30.6 

Unknown 

2 

1.0 

MTF 

Lemoore  NAS 

110 

57.0 

Port  Plueneme 

48 

24.9 

Twentynine  Palms 

17 

8.8 

Fort  Irwin 

15 

7.8 

Edwards  AFB 

3 

1.6 

ENT,  ear/nose/throat;  MTF,  military  treatment  facil¬ 
ity;  VC,  video  teleconferencing. 


Diagnosis  Diagnosis 

Changed  Unchanged 


FIG.  1.  Clinical  impact  of  VC  consultations. 


lated),  this  study  explored  whether  the  overall 
rate  of  clinical  impact  (Fig.  1)  was  general  or 
limited  to  certain  types  of  ENT  problems,  de¬ 
mographic  (e.g.,  age  and  gender),  or  institu¬ 
tional  variables  (e.g.,  military  or  dependent  sta¬ 
tus).  Table  2  shows  rates  of  diagnosis  change 
as  a  function  of  these  variables. 

Table  2  shows  that  the  rate  of  diagnosis 
change  for  the  ENT  consultations  did  not  vary 
by  ENT  problem,  gender,  age,  military  status, 
and  or  facility  site.  No  significant  deviations 
occurred  among  the  major  ENT  categories 
(i.e.,  hearing,  sleep,  tonsil,  upper  respiratory 
lx2(3)  =  1.43,  p  >  0.05].  However,  none  of  the  5 
integumentary  cases  led  to  a  changed  diagno¬ 
sis;  larger  samples  would  be  needed  to  deter¬ 
mine  the  reliability  of  this  latter  trend.  Simi¬ 
larly,  no  significant  differences  were  seen 
within  gender  [x2(l)  —  0.97,  p  >  0.05]  or  mili¬ 
tary  status  [x2(l)  =  1-05,  p  >  0.05]. 

It  may  be  noted  that  active-duty  cases  and 
those  18-64  years  old  showed  rates  of  diagno¬ 
sis  change  similar  to  those  found  in  the  overall 
sample.  The  various  facility  sites  also  appeared 
to  have  similar  rates  of  diagnosis  change,  but 
compliance  problems  and  small  samples  sizes 
from  some  facilities  did  not  allow  significance 
tests.  The  lack  of  substantial  variation  by  these 
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Diagnosis  changed 

No  change 

Overall  summary 

n  =  87 

%  =  45 

n  =  106 

%  =  55 

Type  of  ENT  problem 

Hearing /vertigo 

23 

46 

27 

54 

Sleep 

12 

38 

20 

62 

Tonsil 

13 

46 

15 

54 

Upper  respiratory 

39 

50 

39 

50 

Integumentary 

0 

0 

5 

100 

Gender 

Male 

63 

47 

70 

53 

Female 

23 

40 

35 

60 

Age,  years 

1-10 

6 

32 

13 

68 

11-17 

2 

40 

3 

60 

18-24 

22 

44 

28 

56 

25-44 

43 

47 

49 

53 

45-64 

9 

47 

10 

53 

65+ 

5 

71 

2 

29 

Military  status 

Active  duty 

62 

47 

70 

53 

Dependent 

23 

39 

36 

61 

MTF 

Lemoore  NAS 

48 

44 

62 

56 

Port  Hueneme 

26 

54 

22 

46 

Twentynine  Palms 

5 

29 

12 

71 

Fort  Irwin 

6 

40 

9 

60 

Edwards  AFB 

2 

67 

1 

33 

Totals  less  than  193  cases  reflect  1  or  2  missing  cases  for  certain  vari¬ 
ables  (e.g.  192  for  age  and  gender,  191  for  millitary  status).  VC,  Video 
teleconferencing. 


variables  supports  the  generality  of  the  overall 
rate  of  diagnosis  change. 

Interviews  with  medical  personnel 

Structured  interviews  of  8  physicians  and  7 
nonphysician  medical  personnel  from  the 
TRICARE  Region  9  telemedicine  network  sup¬ 
plemented  the  data  on  patient  outcomes. 


Past  telemedicine  use 

Table  3  shows  the  medical  reasons  for  which 
the  physician  and  nonphysician  medical  person¬ 
nel  said  they  had  ever  used  telemedicine.  The 
primary  reason  given  for  past  telemedicine  use 
was  to  confirm  diagnosis.  All  but  1  of  the  15  med¬ 
ical  personnel  had  at  some  point  used  each  of  the 
telemedicine  technologies  to  confirm  diagnosis. 


Table  3.  Past  Use  of  Telemedicine  Technologies  by  Medical  Personnel 


Telemedicine 

technology 

Physicians  (n 

=  8) 

Nonphsicians  fn 

=  7) 

Diagnosis 
confirmation  (%) 

Education  (%) 

Diagnosis 

confirmation  (%)  Education  (%) 

Telephone 

100 

38 

100 

76 

Fax 

86 

13 

100 

40 

Email 

93 

38 

100 

81 

Internet 

100 

75 

100 

85 

VC 

100 

75 

100 

62 

Overall  average 

95 

48 

100 

69 

VC,  video  teleconferencing. 
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Table  4.  Current  Access  to  Telemedicine  Technologies  by  Medical  Personnel 


Technology 

Physicians  fn 

=  8) 

Nonphysicians  (n 

=  7) 

Current  access  (%) 

Years  used  (M) 

Current  access  (%)  Years  used  (M) 

E-mail 

75 

6.3 

100 

3.0 

RMS 

75 

1.5 

100 

1.7 

Internet 

75 

3.6 

100 

3.6 

VC 

75 

2 

100 

1.7 

RMS,  Referral  Management  System;  VC,  video  teleconferencing. 


The  physicians  and  nonphysicians  differed 
somewhat  in  past  use  of  telemedicine  for  edu¬ 
cational  purposes  (patient  or  provider).  The 
nonphysicians  were  consistently  more  likely 
than  physicians  to  use  telemedicine  for  educa¬ 
tional  reasons.  At  least  70%  of  both  groups  had 
used  the  Internet  to  obtain  medical  informa¬ 
tion. 

Current  access  to  telemedicine  technologies 

Table  4  shows  that  all  nonphysicians  and 
most  physicians  in  the  study  had  current  ac¬ 
cess  to  various  telemedicine  technologies.  All 
participants  reported  well  over  1  year  of  expe¬ 
rience  with  each  technology.  Physicians  and 
nonphysicians  reported  similar  experience 
with  technology. 

Purposes  of  telemedicine  use 

Table  5  shows  the  reasons  physicians  and  non¬ 
physicians  used  telemedicine  during  the  study 
period  (last  4  months).  One  of  the  physicians  was 
excluded  from  this  analysis  for  lack  of  involve¬ 
ment  with  telemedicine  (last  4  months). 

More  than  three-fourths  of  all  telemedicine 
activity  was  initiated  to  confirm  diagnosis.  This 
was  true  for  both  groups.  However,  differences 
between  physicians  and  nonphysicians  were 
observed.  Physicians  were  more  likely  than 
nonphysicians  to  use  telemedicine  for  patient 
education.  In  contrast,  the  nonphysicians  were 


more  likely  to  use  telemedicine  for  their  own 
education  and  for  technical  support. 

Telemedicine  activity  and  satisfaction 

Table  6  summarizes  medical  personnel  re¬ 
ports  of  overall  patient  load  and  telemedicine 
activity  during  the  study  period  (last  4 
months).  Table  7  shows  scores  for  satisfaction, 
usefulness,  and  overall  perception  of  telemed¬ 
icine  technologies  (e.g.,  e-mail,  VC)  and  satis¬ 
faction  with  different  modalities  (images, 
sound,  written  text). 

Level  of  telemedicine  activity 

Nonphysicians  reported  more  telemedicine 
activity  than  physicians,  as  shown  in  Table  6. 
This  is  seen  both  in  the  number  of  telemedicine 
cases  and  in  the  ratio  of  telemedicine  cases  to 
all  patient  care  (telemedicine  cases/telemedi¬ 
cine  plus  non-telemedicine  cases).  This  was  pri¬ 
marily  because  a  nonphysician  at  each  clinic 
was  tasked  to  run  all  telemedicine  sessions 
including  ENT,  neurology,  and  psychiatry.  In 
contrast,  physicians  other  than  the  ENT  spe¬ 
cialist  used  telemedicine  infrequently  in  Region 
9. 11  Substantial  variability  occurred  among 
both  physicians  and  nonphysicians  on  these 
measures,  but  the  sample  sizes  were  quite 
small  (n  =  7,  n  =  7).  To  reduce  the  effect  of  ex¬ 
treme  fluctuations,  median  scores  are  pre¬ 
sented  here. 


Table  5.  Recent  Telemedicine  Usage  by  Medical  Purpose 


Confirm 
diagnosis  (%) 

Patient 
education  (%) 

Provider 
education  (%) 

Technical 
support  (%) 

Unknown  (%) 

Physician  (n  =  7) 

79 

19 

1 

1 

0 

Nonphysician  ( n  —  7) 

77 

4 

9 

9 

1 
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Table  6.  Telemedicine  Activity  Reported  by  Medical  Personnel 


Telemedicine  activity 

Physicians 

Nonphysicians 

(last  4  months) 

(n  =  7)  (Mdn) 

(n  =  7)  (Mdn) 

Telemedicine  patients  seen  (TMED) 

60 

85 

Total  patients  seen  (TMED  +  non-TMED) 

900 

200 

Telemedicine  ratio  (TMED/TMED  +  non-TMED) 

7% 

43% 

Satisfaction  with  telemedicine 

All  personnel  reported  consistently  favor¬ 
able  attitudes  toward  telemedicine  across  all 
types  of  questions/  shown  in  Table  7.  They 
rated  their  attitudes  on  a  5-point  scale,  with 
more  positive  attitudes  indicated  by  lower 
scores  (see  Methods).  Nonphysicians  reported 
only  slightly  more  favorable  attitudes  than  did 
physicians.  Although  the  size  of  this  trend  was 
small,  it  was  seen  for  each  of  the  four  attitudi- 
nal  variables  (technologies,  modality,  useful¬ 
ness,  and  perception).  Each  of  the  four  attitu- 
dinal  variables  consisted  of  five  to  seven 
different  questions.  Nonphysicians  reported 
more  favorable  attitudinal  scores  on  the  ma¬ 
jority  of  questions. 

For  example,  when  answering  questions 
about  their  satisfaction  with  different  tech¬ 
nologies,  the  nonphysicians  indicated  greater 
satisfaction  than  physicians  on  five  of  the  seven 
technologies.  In  particular,  nonphysicians  were 
more  satisfied  than  physicians  with  the  Refer¬ 
ral  Management  System  and  videoconferenc¬ 
ing.  Both  physicians  and  nonphysicians  had 
equally  favorable  ratings  for  store-and-forward 
and  Internet. 

Responding  to  questions  about  satisfaction 
with  specific  modes  of  transmitted  informa¬ 
tion,  nonphysicians  rated  three  of  the  five 
modes  higher  than  did  the  physicians.  The  dif¬ 
ferential  between  the  two  groups  was  particu¬ 


larly  evident  in  reactions  to  sound  and  video 
images.  Both  groups  were  undecided  on  satis¬ 
faction  with  faxed  images.  However,  the  physi¬ 
cians  tended  to  be  more  satisfied  with  e-mailed 
images  than  nonphysicians. 

DISCUSSION 

These  results  support  several  conclusions. 
First,  the  ENT  teleconsultations  had  substantial 
impact  on  diagnosis.  Nearly  half  of  these  tele¬ 
consultations  led  to  a  change  in  diagnosis  by 
the  ENT  specialist  at  San  Diego  relative  to  the 
initial  diagnosis  by  the  primary  provider  at  the 
remote  site.  Importantly,  this  finding  does  not 
vary  by  type  of  ENT  condition,  patient  demo¬ 
graphics,  and  medical  facility  site.  Second, 
medical  personnel  in  the  TRICARE  Region  9 
had  considerable  telemedicine  experience  and 
reported  generally  positive  attitudes  toward 
the  telecommunications  technologies  in  gen¬ 
eral  and  the  telemedicine  process.  Third,  non¬ 
physicians  had  slightly  more  positive  attitudes 
about  telemedicine  than  did  the  physicians. 
This  was  an  interesting  finding,  but  it  is  not  re¬ 
liable,  given  the  small  sample  size. 

Clinical  impact 

It  seems  unlikely  that  the  telemedicine  pro¬ 
cess  itself  (e.g.,  VC)  confounded  the  ENT  ex- 


Table  7.  Telemedicine  Attitude  Scores  Reported  by  Medical  Personnel 


Telemedicine  attitudes 

Physicians 
(n  =  8)  (M)' 

Nonphysicians 
(n  =  7)  ( M ) 

Satisfaction  with  telemedicine  technologies 

1.8 

1.7 

Satisfaction  with  modaltiy  quality 

2.1 

1.9 

Usefulness  of  telemedicine  technologies 

1.6 

1.5 

Perception  of  telemedicine/healthcare 

2.0 

1.7 

Lower  attitude  scores  indicate  more  positive  attitudes. 
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aminations  and  caused  the  specialist  to  change 
the  diagnosis.  The  essential  portions  of  the  ENT 
examinations  required  viewing  video  images 
from  endoscopy  cameras  to  examine  interior  of 
ear,  nose,  and  throat.  This  information  would 
be  the  same  for  "in-person"  or  telemedicine. 
Previous  studies  have  established  high  relia¬ 
bility  between  ENT  telemedicine  and  "in-per¬ 
son"  examinations.20'24  Medical  personnel,  in¬ 
cluding  the  lead  ENT  specialist,  rated  video 
quality  very  highly.  Finally,  the  patients  were 
presented  by  trained  medical  personnel  using 
standardized  protocols.  Therefore,  it  is  reason¬ 
able  to  use  diagnoses  via  VC  as  the  "gold  stan¬ 
dard,"  as  is  standard  practice  in  Region  9,  and 
that  differences  in  diagnosis  in  the  present 
study  reflect  the  patient's  condition,  not  the 
mode  of  consult.  For  this  reason,  no  "in-per- 
son"  control  was  included  in  the  present  study. 

Forty-five  percent  of  ENT  consultations  led 
to  a  change  in  diagnosis.  This  finding  validates 
the  substantial  rate  of  diagnosis  change  (49%) 
reported  in  the  initial  study  of  TRICARE  Re¬ 
gion  9  telemedicine.11  The  data  in  the  previous 
study  were  difficult  to  interpret  because  of 
missing  observations,  which  could  have  biased 
the  observed  rate  in  diagnosis  change.  The 
present  study  provided  a  complete  and  con¬ 
tinuous  sample  of  ENT  consultations  during  a 
planned  4-month  period.  Nonetheless,  the 
present  study  was  limited  to  only  diagnoses 
and  not  outcomes.4  Unfortunately,  these  data 
were  unavailable  in  the  database.  Hopefully,  a 
follow-up  study  will  detail  the  clinical  impact 
of  these  ENT  teleconsultations. 

Several  questions  arise  for  further  research. 
Will  present  telemedicine  procedures  be  cost 
effective  aboard  ships?  This  question  is  espe¬ 
cially  pertinent  because  the  types  of  ENT  prob¬ 
lems  in  this  study  (allergies,  sleep-related) 
could  influence  shipboard  operational  effi¬ 
ciency.  Upper  respiratory  conditions  and  aller¬ 
gies  may  be  a  special  concern  for  small  ships, 
due  to  the  restricted  environments  on  these 
vessels.16  Some  consultations  (e.g.,  tonsillitis- 
related)  could  determine  the  need  for  surgery 
and  evacuation.  Will  the  evaluation  of  consul¬ 
tations  similar  to  those  in  this  study,  conducted 
by  other  ENT  specialists,  extend  the  generality 
of  the  present  findings?  Why  did  some  consul¬ 
tations  lead  to  a  change  in  diagnosis  while 


others  did  not?  Is  it  possible  to  decrease  the 
rate  of  referrals  based  on  the  educational  im¬ 
pact  of  the  consultation  process  for  primary 
providers?  Will  increased  exposure  to  specialty 
consultations  by  the  primary  providers  who 
generate  the  consults  (e.g.,  GMO,  IDC)  lead  to 
improved  diagnostic  skill? 

Attitudes  and  telemedicine  experience 

The  interviews  with  medical  personnel  re¬ 
vealed  generally  favorable  attitudes  toward 
telemedicine  technologies  and  the  telemedicine 
process.  Other  telemedicine  studies  have  fre¬ 
quently  reported  similar  positive  findings.  It 
was  suggested  this  may  be  a  measurement 
problem,  namely  that  patients  and  providers 
were  not  asked  critical  questions.25 

The  present  questionnaire  instrument  was 
designed  to  include  critical  questions.  The  rat¬ 
ings  of  medical  personnel  were  less  favorable 
in  response  to  some  of  these  questions,  such  as 
"Is  the  quality  of  care  rendered  through  the  use 
of  telemedicine  inferior  to  that  provided  in  per¬ 
son?"  Other  questions  that  drew  less  favorable 
responses  were  related  to  satisfaction  with  au¬ 
dio  and  video  and  image  quality  transmitted 
by  fax. 

Because  the  more  critical  questions  did  pro¬ 
duce  less  favorable  ratings,  the  overall  positive 
trend  appears  to  reflect  actual  satisfaction  with 
telemedicine  rather  than  a  measurement  bias. 
Also,  the  trend  for  physicians  and  nonphysi¬ 
cians  to  show  different  patterns  of  response 
suggests  that  the  survey  instrument  had  some 
predictive  validity.  This  study  has  addressed 
several  internal  validity  issues.  Although,  these 
data  were  not  statistically  significant  both  reli¬ 
ability  and  validity  of  the  survey  instrument 
were  plausible.  Further  research  is  indicated, 
however,  to  rule  out  bias. 

The  interviews  showed  that  medical  person¬ 
nel,  particularly  physicians,  were  concerned 
that  telemedicine  might  compromise  patient 
care  in  some  ways  relative  to  in-person  treat¬ 
ment.  In  contrast,  most  of  them  agreed  that 
telemedicine  would  optimize  overall  care  for 
patients. 

ENT  teleconsultations  in  TRICARE  Region  9 
clearly  profited  from  a  functional  infrastruc¬ 
ture,  namely  the  telemedicine  coordinator. 
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medical  personnel,  and  RMS  technology.  One 
key  element  was  the  continuity  of  personnel, 
such  as  the  lead  ENT  specialist,  during  the  past 
few  years.  This  continuity  provided  the  time 
and  expertise  to  work  through  problems  with 
new  technology. 

Physician  and  nonphysician  telemedicine  profiles 

Analysis  of  interviews  contrasted  physician 
and  nonphysician  medical  personnel.  A  small 
but  consistent  pattern  of  differences  between 
these  two  groups  emerged.  Nonphysicians  re¬ 
ported  a  higher  rate  of  telemedicine  use  than 
the  physicians  during  the  present  study  period. 
This  higher  rate  was  due  to  the  fact  that  the 
nonphysicians  often  presented  patients  and  ex¬ 
ecuted  the  telemedicine  sessions  for  a  referring 
provider. 


CONCLUSIONS 

The  present  results  suggest  that  ENT  tele¬ 
medicine  can  have  a  substantial  clinical  impact 
in  military  populations  treated  at  remote  sites. 
Nearly  half  of  the  sessions  studied  (45%)  re¬ 
sulted  in  a  change  of  diagnosis.  This  finding 
did  not  vary  by  age,  gender,  military  status, 
ENT  conditions,  or  treatment  facilities.  In  ad¬ 
dition,  the  medical  personnel  reported  very  fa¬ 
vorable  attitudes  toward  telemedicine  tech¬ 
nologies  and  the  telemedicine  process.  Thus,  it 
may  be  worth  investigating  the  possibility  of 
utilizing  ENT  telemedicine  for  shipboard  med¬ 
ical  departments.  Substantially  reducing  the  in¬ 
cidence  of  ship-to-shore  medical  evacuations 
could  significantly  impact  operating  costs,  de¬ 
spite  the  cost  of  telecommunication  bandwidth 
requirements.6 
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